Conventional blade servers. FIG. 1 is an exemplary illustration of a conventional blade server 100 connected to an external switched fabric. Blade servers overcome some of the inefficiencies of individual standalone or rack-mounted one unit (1U) high servers, each of which is self-contained and includes separate power supplies, fans, and the like. Individual servers are therefore inefficient in terms of space, power, cooling, and other characteristics. Blade servers 100 utilize a modular, plug-in approach wherein the housing for each server is eliminated along with self-contained components such as power supplies and fans. Each previously standalone server is therefore reduced to a server “blade” 102 (typically eight to 14 in a blade server chassis 106) capable of being plugged into a midplane 104 within the blade server chassis 106 from the front of the chassis. The midplane 104 contains connectors for receiving the server blades 102 and typically contains from one to four “lanes” or paths on a printed circuit board (PCB) for carrying signals. The midplane 104 therefore eliminates much of the cabling that was required with individual servers. The blade server chassis 106 also provides redundant common cooling and power to the server blades 102 through the midplane 104.
Conventional blade servers 100 may be connected to redundant external switch fabrics 108 through an “A” side Input/Output (I/O) switch 110 and a “B” side I/O switch 112, which plug into the midplane 104 from the back of the chassis 106. Typically, the redundancy enables one switch to take over if the other fails. In addition, the blade server midplane is typically plumbed to allow for multiple independent redundant fabrics or I/O protocols, such as Fibre Channel, Ethernet or InfiniBand. In the case of a Fibre Channel configuration, each embedded switch 110 and 112 may be a Fibre Channel Arbitrated Loop (FC_AL) switch or a full fabric switch, with a separate port to receive a Fibre Channel (FC) link 116 from each of the multiple server blades 102, and output ports for connecting to each of the external switched fabrics 108.
To enable the server blades 102 to communicate with the switch fabric, typically a mezzanine I/O card 114 that performs a Host Bus Adapter (HBA) (a.k.a. I/O Controller (IOC)) function is required in each server blade 102. These mezzanine I/O cards 114 are typically mounted to the server blades 102 as daughter cards. Note that this may also be accomplished by embedding an IOC directly on the server blade. However, this increases complexity for the OEM, who must now make a different server blade for each type of I/O that will be supported. For purposes of this specification, mezzanine I/O cards 114, referred to herein, include both daughter cards and IOCs mounted directly onto the server blade. The output of a mezzanine I/O card 114 is two I/O links 116 routed to each of the two embedded switches 110 and 112. The mezzanine I/O cards 114 follow the standard device driver model, so that when a server blade 102 with a mezzanine I/O card 114 is plugged into the midplane 104 and connected to an embedded switch 110 or 112, it appears to be a standalone server with a Peripheral Component Interconnect (PCI) card communicating with an external switch.
As a means of delivering I/O, mezzanine I/O cards 114 are relatively expensive (compared to the cost of a server blade 102), take up additional space in the rack, generate additional heat, and consume additional power. Because space efficiency, low power and lower heat are significant value propositions for blade servers, the use of mezzanine cards limit the appeal of the overall blade server solution. A key drawback to mezzanine cards has to do with serviceability. Because the mezzanine card is installed in parallel with the server blade, if a problem should arise with the mezzanine card, the entire server blade must be removed to troubleshoot and/or replace the mezzanine card, creating server downtime costs for end users.
In addition to the problems introduced by mezzanine I/O cards 114 described above, the blade server concept also constrains I/O connectivity because it is slot limited. Conventional standalone 1U servers have two PCI slots to plug in two different types of I/O cards. However, in the blade server environment, the server blades 102 have no slots and can typically only accommodate one mezzanine card. Consequently, blade servers that require more than one type of I/O protocol must typically embed an IOC on the blade server itself, in addition to deploying a mezzanine card. For blade servers that require multiple I/O protocols to support multiple network applications, this limitation has created a need for a next generation blade server I/O architecture that can flexibly support multiple I/O protocols.
The use of embedded switches, if they are fabric switches, can result in a violation of the so-called “hop” limitation. Some storage environments do not allow more than three fabric switches to be present in the path between a storage array and a server blade. This limitation on the number of fabric switches is called the hop limitation. For example, with a 3-hop limitation, only three fabric switches can be present in the path between the storage array and the server blade. Adding an embedded fabric switch to the blade server chassis 106 can potentially violate the three switch limit.
Furthermore, interoperability issues may exist between the embedded switches 110 and 112 with E_PORT connectivity and the external switched fabrics 108. For example, the embedded switches of one manufacturer may only interface properly with external fabric switches made by that same manufacturer. These interoperability limitations may force blade server manufacturers to qualify, certify and stock a number of different types of embedded switches 110 and 112. These interoperability issues can be overcome with N_PORT virtualization, discussed below.
Emerging next generation blade server system interconnect. Conventional blade servers such as those shown in FIG. 1 primarily utilize parallel PCI in the midplane. However, PCI Express, which is the serialized counterpart of PCI, is now replacing parallel PCI, and can be routed over lanes in the midplane. Referring now to FIG. 2, proposed next generation blade servers 200 will route PCI express 202 over the midplane 204, enabling the IOC function to be moved to a separate location on the chassis away from the blade server itself, eliminating the need for mezzanine I/O cards directly embedded on the server blades 206.
The concept of shared I/O, which deploys an IOC 212 that is shared by multiple hosts (server blades 206) has also emerged to address mezzanine card challenges in blade server environments. Utilization of shared IOCs 212 can result in substantial space savings. In conventional systems, each server blade is connected to a dedicated IOC, and therefore a conventional blade server may include a 14 blade chassis with 14 discrete IOCs (i.e. mezzanine cards) and redundant embedded switches. Using shared I/O, the 14 discrete IOCs may be replaced with fewer shared IOCs, each of which has multiple different frontside “personalities” for the multiple server blades to which it may be connected. However, in order to share the IOC, a new switch must be introduced between the Shared IOC and the blade server. These Shared I/O connections 216 will be logically shared by the server blades 206 connected to the Shared IOC 212 to enable them to communicate with an external switched fabric 214. Because the conventional I/O driver model assumes that each server has a dedicated IOC that is always available, the Shared I/O model requires a change in the I/O driver model, which creates incompatibilities with existing network I/O applications and adds driver complexity to end-user datacenter installations.
To implement shared I/O, the driver model may be changed so that the server blades cooperatively share the IOC 212. However, this latter approach would require new software that would be incompatible with the installed base of I/O drivers to which many applications interface. Furthermore, the shared IOCs 212, although conceptualized, have not yet been implemented. Therefore, although the cost of a shared IOC may be comparable in price to discrete IOCs, the cost and space savings would have to be traded off with the cost of deploying an entirely new switching infrastructure between the blade server and the Shared IOC.
PCI Express switches or Advanced Switch Interface (ASI) (peer-to-peer network) switches 208 and 210 may be deployed between the blade server and the Shared IOC to enable Shared I/O. As FIG. 2 illustrates, in Shared I/O the switches 208 and 210 are located in front of the remote IOCs 212. ASI is a superset of PCI Express. It uses the same electrical signaling, but also provides an endpoint termination device that allows peer-to-peer communication. ASI has developed semantics similar to InfiniBand, which is a low latency peer-to-peer short distance network that is not designed to compete with Ethernet, but rather is designed to compete with older clustering interconnects. Thus, ASI is a true networking device as well as a bus. PCI Express serializes the parallel bus, but is otherwise semantically identical. However, like shared IOCs, these physical layer switches have been conceptualized but not yet implemented.
From a server blade perspective, when changing to a next generation device, it is preferable to utilize the same driver software model so that the same applications software can be executed and the complexity of the network is minimized. However, the Shared I/O model of FIG. 2 will not achieve this compatibility, because new driver software is needed to logically associate the characteristics of the shared IOCs 212 with the server blades 206. In addition, it may be desirable for each server blade 206 to have multiple instances of shared IOCs 212 available for it, because it has a heavier I/O load than another server blade 206. This load balancing requires a new dynamic management mechanism to associate and disaggregate the shared IOCs 212 with the server blades 206.
Therefore, there is a need to provide the benefits of shared I/O in a blade server that achieves similar space, heat, power, serviceability and cost savings, but does not require new blade server driver software, a new shared IOC design, a new switching infrastructure, new consolidated physical layer switches, and new management software.